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(54) Image motion vector detecting device and swing correcting device 



(57) In image motion vector detecting device has a 
part (3) for determining correlation values in specific 
deviation in each detecting region of disposed plural 
detecting regions in a screen, a part (4) for determining 
a time variation value of the correlation values, a part (2) 
for determining a motion vector in each detecting region 



from the correlation values, a part (5) for judging a relia- 
bility of motion vector in each detecting region by using 
the correlation values and the time variation value, and 
a part (6) for determining a motion vector of an entire 
screen by using the motion vector of each region on the 
basis of the judged reliability. 
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Description 

BACKGROUND OF THE INVENTION 
5 1. Field of the Invention 



1 Field or tne invenuun . . 



due to shake or swing of video camera 
2. Description o1 the Prior Art 
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escnption oi ine rnuin,. 

as follows. n 

Correlation value = £|AUCm) 
k-1 

where 
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n : number of subregions in one det.ect.ng reg.on 
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n : numoer oi mw«m"«» o1 each 

comparator 220, and are set value, « is judged that the rel.ab.hty « fe*ng (reja 

steD 222 corresponds to step <iu<* »■ n «' ' ^ 

Jon vector «^^S^„Hn^^^r?-^S^^ 

The swing correcting means *>o fluctuation is fed as shown in ^o, » 

the reading pli«ons of theses^ 

of the n-1 -th field, the n-th f .eld, and ^ e ^ e ™"]° a size of one screen less in fluctuaton. 

5 J^^*-* — ^^M^Lna J of the periphery of the n— 
minimum of the correlation values * I**™ ^ oi the corre iatjon values. 
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present state of the cor elation values such as the minimum, mean and maximum. However, since the correlation value 
at each time is unstable due to noise or the like, the reliability cannot be correctly judged by the correlation value at each 
time alone. 

Besides, as in the prior art, when judging on the binary basis (0 or 1) to determined the reliability by a predeter- 

5 mined threshold level, rf the reliability judgment in a certain range of the threshold level of the judging condition is ambig- 
uous, in other words, if there is an uncertain region of reliability judgment in a width of w1, w2 with respect to the 
threshold levels of condition 1 , condition 2 as shown in Fig. 6, the following case may be considered. Suppose there are 
two sets of data D1, D2 as shown in Fig. 6. Both D1 and D2 were obtained in the same condition, but were unstable 
due to noise or the like, and were obtained as different data. By nature, these two are same data, and are positioned at 

w a very close distance, but in the ambiguous region as for condition 2, and hence D1 is judged to be reliable, and D2 not 
reliable. As for condition 1, however, the data D1.D2 belongs obviously to the reliable region. In this case, therefore, it 
is correct to judge that both D1 and D2 are reliable, but actually D2 is misjudged to be not reliable. 

Due to such cause, the motion vector detecting device of the conventional constitution often misunderstood that a 
moving object entered the screen although it was not actually, or detected a vector added with the motion of a moving 

is object when the moving object entered the screen. Therefore, the swing correcting device often malfunctioned as the 
screen was corrected in a wrong direction from the direction intended by the operator as the correction was interrupted 
or a vector added with the motion of the moving object was detected. 

In the conventional constitution of the motion vector detecting device, moreover, the motion vector detecting device 
often malfunctioned when a moving object entered the screen because the motion vector of the entire screen was 

20 judged from the result of the determination of the reliability in each region. That is, detection of a moving object is done 
independently in each region, a sufficient detecting performance of moving object cannot be obtained. 

This point is described in detail below. Fig. 7 (a) shows the time course variations of the correlation values when a 
moving object enters the screen, and Fig. 7 (b), when vibrations of a specific period are applied to a camera. The period 
indicated by T1 in Rg. 7 (a) is the period when the moving object is present, and the period indicated by T2 in Fig. 7 (b) 

25 is the time of applying vibrations to the camera. To distinguish the motion due to the moving object and the motion due. 
to camera swing, the level of the correlation value is noticed. In the case of the moving object the correlation value 
increases largely due to invasion of the moving object, but the fluctuation of the correlation value is small in the case of 
camera swing. Accordingly, when the correlation value is larger than a certain level, it is regarded to be due to the mov- 
ing object, and the reliability is judged to be absent Depending on the circumstances, however, the correlation varies 

30 very little due to moving object or the correlation varies very much due to camera swing, and then it is difficult to cor- 
rectly judge the two in each region. 

Incidentally, such point that when a moving object enters the screen, the motion vector added with the motion of 
the moving object is detected, is described in detail below. In this case, the swing correcting device often malfunctions 
by correcting the screen in a wrong direction from the direction intended by the operator. 

35 Referring now to the drawings, this is explained more specifically. As shown in Fig. 64, while the video camera is 
stopped still, suppose a moving object 52 passes through the screen. First, as shown in interval (a) in Fig. 64, when the 
background and the moving object coexist within a certain detecting region 48 .this detecting region 48 is judged to be 
not reliable by the state of correlation value 53. On the other hand, the still background in a detecting region 49 is 
desired to be judged reliable by reliability judging means , but depending on the screen appearance of the background, 

40 it may be misjudged to be not reliable as the correlation value 54. 

Next, as shown in interval (b) in Fig. 64, when occupying the majority of the detecting region 48, the motion vector 
expressing the motion of the moving object is detected from that detecting region, and a smaller value 55 than the spec- 
ified value is indicated by the correlation value 53, and the reliability judging means judges reliable. Besides, the back- 
ground of the detecting region 49 is still judged to be not reliable, and as a result the motion of the moving object is 

45 employed as the motion vector of the entire screen, and when the swing is corrected on the basis of this, the still back- 
ground is moved, which results in an unnatural image (the operation of the detecting regions 50, 51 is same as that of 
48, 49, respectively, and is not explained herein.). 

Fig. 8 shows the vector of each detecting region and the motion vector of the entire image at this time. A vector 234 
corresponds to the vector of the moving object of 52 of Fig. 64, a vector 235 to the vector of the detecting region of 48 

so of Fig. 64, a vector 236 to the vector of the detecting region of 49 of Fig. 64, a vector 237 to the motion vector of the 
entire screen, and a vector 238 to the vector of the background. Intervals (a) to (e) in Fig. 18 correspond to intervals (a) 
to (e) in Rg. 64. The motion vector detected in intervals (a) to (e) in Fig. 64 is as shown by 237 in Rg. 8, and when the 
swing is corrected on the basis of this vector, an abnormal correct ion occurs suddenly in the state of interval (b) in Fig. 
8. Furthermore, the vector due to the camera swing may be added to all vectors in Fig. 8, and an abnormal correction 

55 may occur due to the same reason. 

In the conventional motion vector detecting device, still more, when the screen is swinging and it is being corrected, 
the reliability judging means suddenly judges not reliable depending on the screen appearance, and 0 vector is often 
delivered from the vector determining means. As a result, the swing correcting device does not function, and the screen 
shows a swinging, hard-to-see image. 
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Practical examp.es aredescribed below by reference to Figs. 71 and* Fig. 71 ^^^^""^J. 
exoreSng the camera swing, detecting regions 75, 77. their correlation value 80. detecting regions 76 78 and the*: 
Son valued ( the arfan of the detecting regions 77, 78 is same as that of 75 76. respectively, and k not 
eSed ) Numeral 79 is an obstacle. To begin with, in intervals (a) and (c) in Fig. 71. depend.ng on the stete of the 
£2ton values 80 81 the reliability is affirmatively judged in the detecting regions 75. 76, and swmg may be cor- 

ground 3I?*£t are present in a certain detecting region 75, depending on the state of the correlat.cn vak* 
lo the deVeS region 75 is judged to be not reliable. On the other hand, the background of the detecting region 76 is 

screen swing is corrected, and a normal motion is achieved. 

Fig. 9 shows the motion vector of the intervals in Fig. 71 . In Rg. 9. numeral 241 is a motion' ^ rt J*?25 
which swings vertically about 0 vector. Numeral 240 is a vector detected from a detecting region 75 and a motion vector 
2 ^ duTcamera Jng.amo«on vector of the m0 ^«^WInFi B .71 oro^«««v^-^^ 
239isavector detected from a detecting region 76, ^^ M ^ M ^^ a ^^^Si 
242 isamotion vector ol the entire screen delivered from a vector ^^^^^^^^ 
(d) in Fig. 71 correspond to the intervals (a) to (d) in Fig. 81 . Same as ,n Fig. 71 . ,n .ntarod ^ ^T^H. 
the detecting regions become 0. and the motion vector 242 ol the entire screen is also 0. As a result, suddenly, the 
swina is not corrected, and the screen shows a swinging, hard-to-see image. 

in the ^seTmuitiple vector detecting regions, malfunction of vector detection when a moving objects enters the. 

T 52SS1fw W. suppose a moving object (a car) 246 passes through the screen while the video cam- 
« -T^^H^^itS^i region, 244 is a motion vector of the background, and 245 « a motion vector 
a ^S^TSSS^SSIU to be reliable by the reliability judging mean, and me correla«on value 
249 stow^a smaner value 250 than the specified value. While the background and moving object coexist, since the cor- 
relation^lue ^^fdo« no^ss the'specrfied minimum value, and rts detecting region is judged to be not reliable 
S shaded area in Fig 10) When the motion of the moving object has occupied the majority of the detecting regna 
Semoti^ 

a«on value ^shows asmaller value 248 than the specified value, and the reliability is affirmatively judged by the reh- 
S iSiS I m STiHilt both the motion of the moving object and the motion of the background are employed 

object is contained, an unnatural image is produced. 



35 SUMMARY OF THE INVENTION 

To solve the above problems, the invention presents the following constitution. 
An image motion vector detecting device of the present invention comprises; 

« means for determining correlation values in specif ic deviation in each detecting region of disposed plural detecting 
regions in a screen, 

means for determining a time variation value of the correlation values, 

means for determining a motion vector in each detecting region from the correlation values 

means £ fudging a reliability of motion vector in each detecting region by using the correlate values and the t,me 

" ^SiSShfl a motJon vector of an entire screen by using the motion vector of each region on the basis 
of the judged reliability. 

An image motion detecting device of the present invention comprises; 

means for determining a motion vector in each detecting region of disposed plural detecting regions in a screen 

pieced Tof reliability judgment information which is obtained by using .correlation va,u «'"^ ^^^^ 
judgment value converting means for converting the judgment result havmg a meaning from true to false into binary 

SS SSSrt^S J mSntctor of an entire screen by using the motion vector of each region on the basis 
of the binary judgment value of each detecting region. 

An image motion vector detecting device of the present invention comprises; 
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means for determining a motion vector in each detecting region of disposed plural detecting regions in a screen, 
judging means for determining a judgment result having a meaning from true to false by a reliability judgment infor- 
mation which is obtained by using correlation values in the each detecting region , and ™.7-\ 

means for determining a motion vector of an entire screen by using the motion vector of each region on the basis 
5 of the judgment result having a meaning from true to false . 

An image motion vector detecting device of the present invention comprises; 

means for determining a motion vector in each detecting region of disposed plural detecting regions in a screen, 
w means for determining a correlation of motion vectors among the detecting regions, 

means for judging a reliability of the motion vector in each detecting region by using the correlation values, 
means for changing a judgement condition of the reliability judging means by a correlation value of motion vectors 
among detecting regions, and 

means for determining a motion vector of an entire screen by using the motion vector of each region on the basis 
is of the reliability judgment. 

An image motion vector detecting device of the present invention comprises; 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect 1 
20 ing regions in a screen, 

means for determining a difference of correlation values between right and left, or upper and lower detecting 
regions in a screen, 

means determining a motion vector in each detecting region on the basis of the correlation values, 
means judging a reliability of the motion vector in each detecting region on the basis of the correlation value in each 
25 detecting region, and 

means determining a motion vector of an entire screen by using the motion vector of each detecting region on the 
basis of the difference of the correlation values between regions and the judgment of reliability. 

An image motion vector detecting device of the present invention comprises; 

30 

means for determining a correlation value in a specific deviation of each detecting region of disposed plural detect- 
ing regions in a screen, 

means for determining a difference of correlation value between right and left, or upper and lower detecting regions 
in a screen, 

35 means for determining a motion vector in each detecting region on the basis of the correlation values, 

means for judging a reliability of a motion vector in each detecting region on the basis of the correlation value in 
each detecting region and the difference of correlation values between regions, and 

means for determining a motion vector of an entire screen by using the motion vector of each detecting region on 
the basis of the judgment of the reliability. 

40 

An image motion vector detecting device of the present invention comprises; 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect- 
ing regions in a screen, 
45 means for determining a time variation value of correlation values, 

means for determining a difference of the time variation values of the correlation values between right and left, or 
upper and lower detecting regions in a screen, 

means for determining a motion vector in each detecting region on the basis of the correlation values, 
means for judging a reliability of motion vector in each detecting region on the basis of the correlation value in each 
so detecting region and the time variation value of the correlation values, and 

means for determining a motion vector of an entire screen by using the motion vector of each detecting region on 
the basis of the judgment of reliability and the difference in time variation value of correlation values between 
regions. 

55 An image motion vector detecting device of the present invention comprises; 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect- 
ing regions in a screen, 

means for determining a time variation value of the correlation values, 
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the basis of the reliability. 

ina regions in a screen, ^ + ^; n n renion bv the correlation values, 

vector or the correlation value or their combination. usi lhe motion vector. 

basis of the obtained final judgment result 

mtheimage motion vector detecting device of the present invention ; 

motion vectors led out in the time ser ieS wrthm a spec-f.c range. 

aco— va.ueinaspec.cd^onineachdete.ngr^ofdisposedplu.detect- 

ing regions in a screen, HptPrtina reaion by the correlation values. 

of the obtained final judgment result. 

,n the image motion vector detecting device of the present invention; 

• h ■«« means iudges only one minimum va,ue among the motion vectors of detecting region, 
the minimum judging means juages omy u. 

inq regions in a screen, j rt+a ,*tinn rpninn from the correlation values, 

basis of the obtained final judgment result 
In the image motion vector detecting device; 
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the state judging means is a reliability judging means for judging a reliability of the motion vectors in detecting 
regions by the motion vector value or the correlation value or their combination. 

An image motion vector detecting device of the present invention comprises; means for determining a correlation 
5 values in a specific deviation of each detecting region of disposed plural detecting regions in a screen, 

means for determining a motion vector in each detecting region from the correlation values, and 

weighting processing means for determining a motion vector of an entire screen from the motion vectors and weight 

coefficients set in each detecting region . 

w 

An image motion vector detecting device of the present invention comprises; 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect- 
ing regions in a screen, 

75 means for determining a motion vector in each detecting region from the correlation values, 

reliability judging means for judging a reliability of the motion vector of detecting regions by the motion vector value 
or the correlation value or their combination, and 

weighting processing means for determining a motion vector of an entire screen from weight coefficients set in 
each detecting region and the motion vector of the detecting region Judged to be reliable by the reliability judging 
20 means . 

An image motion vector detecting device of the present invention comprises; 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect- 
25 ing regions in a screen, 

means for determining a motion vector in each detecting region by the correlation values, 
classifying means for classifying the motion vectors into groups on the basis of a magnitude or direction or their dif- 
ferential value, 

camera shake group judging means for selecting a group for determining a motion vector of an entire screen from 
so groups classified by the classifying means, and 

means for determining a motion vector of the entire screen from the motion vector of the group selected by the cam- 
era shake group judging means. 

An image motion vector detecting device of the present invention comprises; 

35 

means for determining a correlation value in a specific deviation in each detecting region of disposed plural detect- 
ing regions in a screen, 

means for determining a motion vector in each detecting region from the correlation values, 

reliability judging means for judging a reliability of a motion vector of the detecting region by the motion vector value 

40 or the correlation value or their combination, 

classifying means for classifying the motion vectors of the detecting regions judged to be reliable by the reliability 
judging means into groups on the basis of a magnitude, direction or their differential value, 
camera shake group judging means for selecting a group for determining a motion vector of an entire screen from 
a group classified by the classifying means, and 

45 means for determining a motion vector of the entire screen from the motion vector of the group selected by the cam- 
era shake group judging means. 

An image swing correcting device of the present invention comprises; 

so image signal memory means or CCD for at least one field, 

said above-mentioned image motion vector detecting device , and 

reading position of the image signal memory means or CCD being controlled on the basis of a motion vector 
detected by the image motion vector detecting device. 

55 The invention, having the constitution as described herein, judges the reliability of the vector in each detecting 
means by using not only the state of the correlation value in each detecting region, but also the time-course variation of 
the correlation value, Therefore, if the absolute value of the correlation value at a certain time is fluctuated by the noise 
and the reliability cannot be judged correctly, the judgment may be compensated by paying attention to the time-course 
variations. 
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Vtttt&Htt^XEZ^ '" — * ■ "~ * 

a»»d if TO one ol pta.l iudg™rt isgons s OThguoui ™^ng m*ig coficllon ol 

o correlation value and the time-course ^^^J^^^i the moving object isdetected. and the motion 
regions °r the difference in time-course vana^ < ^ ^ ^ ^ entire 

vector ol the entire screen is determ.ned. Furthermore o.v he basisrt in » ^ 
screen, by contro..ing reading position of ^^J^r^^Z 7«£ in each detecting region is 
incidentally, when the presence of . riot ™r™!£££Z ^background, regardless of the result of the 
„ judged by the still judging means, since ' « ^ « JiJ* still vector. 

reliability judgment, the motion vector of *^X^^SSS^ * '«* in one deteCtin9 ,e9 '° n ' *" "!? " 

As a result of judgment ot rel.ab.lrty if- »» " "'£3^ is regar ded as the still background, and it * 

vector of a relatively small value judged by the d reliability. 

obstacle should not be used in correction operation. ft t of the screen denotes the motion of the 

Aweight coefficient is set in each deW 

25 subject, ^^''^^^T^A^^^r!^^ basis of this weight coefficient, when the 

:Xh^^^ 

bya S~edete.ing ^^^^^^^S^!^ 

3 o motion vector of the subject, and when ™J™**£™™ J^!w shake is further enhanced, 
ing regions judged to be reliable the cor^ 

As mentioned in relation to the problems ftat £J the motion of the subject coexist. Furthermore, the 
by the reliability judging means, the motion of the camera shake and l£ mow ^ ^ ^ ^ 

Motion of the subject may not be 

be extracted. . t , ble by me reliability judging means, since the motion vec- 

BRIEF DESCRIPTION OF THE DRAWINGS 

r 1 • aHockdiaaramof a swing correction device applying a 
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Fig. 22 is a block diagram of an image motion vector detecting device in a fifth embodiment of the present invention. 

Fig. 23 is a detailed structural diagram of reliability judging means and judgment converting means in the fifth 
embodiment the present invention. . - 

Fig. 24 is a diagram for explaining the operation of the reliability judging means in the fifth embodiment of the 
present invention. 

Fig. 25 is a block diagram of a swing correcting device in a sixth embodiment of the present invention. 
Fig. 26 is a block diagram of an image motion vector detecting device in a seventh embodiment of the present 
invention. 

Fig. 27 is a block diagram of a swing correcting device in an eighth embodiment of the present invention. 
Fig. 28 is a block diagram of an image motion vector detecting device in a ninth embodiment of the present inven- 
tion. 

Rg. 29 is a detailed structural diagram of motion vector determining means in the ninth embodiment of the present 
invention. 

Rg. 30 is a detailed structural diagram of vector weighting means 17 in the ninth embodiment of the present inven- 
tion. 

Rg. 31 is a block diagram of a sing correcting device in a tenth embodiment of the present invention. 
Rg. 32 is a block diagram of an image motion vector detecting device in an eleventh embodiment of the present 
invention. 

Rg. 33 is a block diagram of a swing correcting device in a twelfth embodiment of the present invention. 
Rg. 34 is a block diagram of an image motion vector detecting device in a thirteenth embodiment of the present 
invention. 

Fig. 35 is a block diagram of a swing correcting device in a fourteenth embodiment of the present invention. 
Fig. 36 is a block diagram of an image motion vector detecting device in a fifteenth embodiment of the present 
invention. 

Fig. 37 is a diagram for explaining a changing method of judging condition. 

Fig. 38 is a block diagram of a swing correcting device in a sixteenth embodiment of the present invention. 
Rg. 39 is a block diagram of an image motion vector detecting device in a seventeenth embodiment of the present 
invention. 

Fig. 40 is a block diagram of a swing correcting device in an eighteenth embodiment of the present invention. 
Rg. 41 is a block diagram of an image motion vector detecting device in a nineteenth embodiment of the present 
invention. 

Rg. 42 is a block diagram of a swing correcting device in a twentieth embodiment of the present invention. 
Fig. 43 is a block diagram of an image motion vector detecting device in a twenty-first embodiment of the present 
invention. 

Rg. 44 is a block diagram of a swing correcting device in a twenty-second embodiment of the present invention. 
Fig. 45 is a block diagram of an image motion vector detecting device in a twenty-third embodiment of the present 
invention. 

Rg. 46 is a block diagram of a swing correcting device in a twenty-fourth embodiment of the present invention. 
Fig. 47 is a block diagram of an image motion vector detecting device in a twenty -fifth embodiment of the present 
invention. 

Rg. 48 is a diagram for explaining the operation of the motion vector detecting device in the twenty-fifth embodi- 
ment of the present invention. 

Rg. 49 is a block diagram of an image motion vector detecting device in a twenty-sixth embodiment of the present 
invention. 

Fig. 50 is a block diagram of an image motion vector detecting device in a twenty-seventh embodiment of the 
present invention. 

Fig. 51 is a diagram for explaining the operation of the motion vector detecting device in the twenty-seventh embod- 
iment of the present invention. 

Fig. 52 is a block diagram of an image motion vector detecting device in a twenty-eighth embodiment of the present 
invention. 

Rg. 53 is a block diagram of a swing correcting device in a twenty-ninth embodiment of the present invention. 
Fig. 54 is a block diagram of an image motion vector detecting device in a thirtieth embodiment of the present inven- 
tion. 

Fig. 55 is a diagram showing an example of detecting region of a motion vector. 

Fig. 56 is a diagram for explaining the operation of the motion vector detecting device in the thirtieth embodiment 
of the present invention. 

Rg. 57 is a diagram showing an example of a detecting region of a motion vector. 

Rg. 58 is a block diagram of an image motion vector detecting device in a thirty-first embodiment of the present 
invention. 



EP 0 789 487 A2 

Fig . 59 isa W ocKdiagram^ 
^ ^60 is a blocK diagram of an image motion vector detecting device in a frirty-third e^odi.ent ^e present 
'"Tel is a block diagram of an image motion vector detecting device in a thirty-fourth embodiment o. the present 
inV Ce2 is a b.ocX diagram of an image motion vector detecting device in a thirty-fifth embodiment of the present 
""To!..!*, chart of 

Flg . 65 is a diagram for ^^SSSSSSSSt of the screen and the mean value. 
j5 ment ol the present invention. H «t a r*inn ripvice 

of the present invention. , . 

F,g. 74 is adiagram showing a detecting , fegK,r ' '"^eea |r Rg ?4 

°' T'tirXam of invasion of the moving object (automobile) into the detecting region, (b) is a diagram 

*" ?j£ Sciagram of invasion of a moving object (a man) invading in the detecting region, (b) is a diagram of 
clustering of motion vectors in each detecting region^ 

tered detecting regions. 
, DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION 

45 of vector, and 6 is motion vector detecting means. embo diment. first of all, an image signal contin- 

,n thus combed in^ger^tionvectordetect^ 

region, preliminarily, as shown m 209, 210, 211. m m "0 • m o{ sition signa is deviated by the 
screen, and concerning the detecting reckons between toe ifmkfe the ™ ? , ation ^ zmu 

so quantity in specrfied range(ij) :imin < i < imax , ,m,n < < .max „ obt ^ ned as me m otion vector of 
.and the deviation^') for giving the m,n.rnum value of that value * ^ yalues are 

each region. In the correlation value detecting ^ ^^^ZiS^^ differentia, value after averag- 
obtained. In the time-cou.se var.at.on detecting Pf .^J^SSSy judging circuit means 5 of vectors judges 
ing the correlation values, or th. ^^^^^tS!^ va.ue detecting part 3 and the time- 

55 thereliabilityineachreg^ 
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the quantity in an even number, the mean of two values close to the middle position ) as the motion vector of the entire 
screen, and delivers. 

What is different from the conventional motion vector detecting device is that tile reliability of the? motion vector of 
each detecting region is judged not only by the state of the present correlation value in each detecting region, but also 

5 by the time-course variation value of the correlation value. This point is described in further detail. As shown in Fig. 2. 
when a moving object such as a person enters the vector detecting region, the time-course changes of the correlation 
value are shown in Fig. 12 and Fig. 13. In both Fig. 12 and Fig. 13, (a) is the minimum of the correlation value, and (b) 
denotes the time-course variation ( differential value) of the minimum of the correlation value. At this time, as the judging 
level of the reliability, supposing the minimum to be level m, and the time-course change of the minimum to be level b, 

10 the reliability is negative if at least one of them is greater than the judgment level. Fig. 12 and Fig. 13 are photographs 
of almost the same scene, and the moving object has crossed before the camera in both, and the minimum is near the 
judgment level. In (a), the minimum is over the level m, and the invasion of the moving object detected and the reliability 
is negatively confirmed. On the other hand, in (b) the minimum is less than level m, and the invasion of the moving 
object cannot be detected by the minimum alone, but the differential value of the minimum is over level b, and therefore 

is invasion of the moving object can be detected. 

Thus, according to the embodiment, when judging the reliability of the motion vector in each detecting region, it is 
possible to correctly judge the reliability by detecting not only the state of the present correlation values of the detecting 
regions, but also the time-course variations of the correlation values, even if one of them is fluctuated by noise or the 
like,. 

20 A block diagram of a swing delecting device in a second embodiment of the present invention is shown in Fig. 14. 
The same parts as in Fig. 1 1 are identified with same reference numbers, and repeated explanation is omitted. In Fig. 
14, numeral 7 is swing correcting means. 

In thus composed second embodiment, the blocks of the image motion vector detecting device are same in oper- 
ation as the first embodiment. 

25 The swing correcting means 7, when an image of a large fluctuation enters as shown in 226, 227 in Fig. 5, moves 
the image parallel from 229 to 229 in the direction 231 of correcting the motion, on the basis of the motion vector 230 
entered from the motion vector determining means 6, as to the input image signals of the (n-1)-th field 226, and the n- 
th field 227, and expands the screen by interpolation or the like, and delivers image signals 232, 233 in a size of one 
screen small in swing. 

30 Thus, according to the embodiment, when judging the reliability of the motion vector of each detecting region, by 
using not only the state of the present correlation values in the detecting regions but also the time-course fluctuation 
values of correlation values, the reliability may be judged correctly even if one of the values is fluctuating due to noise 
or the like, and even if a moving object occupying the majority or whole of the screen passes through the screen , mal- 
function to correct the screen in a different direction from the intended direction of the operator due to interruption of 

35 correction does not occur. Meanwhile, the number of detecting regions in the screen of the first and second embodi- 
ments is four, but it is not limitative. 

A motion vector detecting device in a third embodiment of the present invention is described below. The configura- 
tion of the motion vector detecting device in the third embodiment is shown in Fig. 15. Those same as in Fig. 11 are 
identified with same reference numbers and are not explained. In Fig. 15. numeral 8 is a reliability judging information 

40 detecting part, 9-1 9-n are first judging means for determining the judgment value of ternary or higher value taking 
the values in interval [0, 1], 10 is second judging means for determining the judgment value of the ternary or higher 
value taking the values in the interval [0, 1] from the judgment values of plural ternary or higher values by the first judg- 
ing means 9-1,9-2 9-n, and 1 1 is judgment value converting means for converting the judgment value by the second 

judging means 10 into a binary judgment value of {0, 1}. In this embodiment, as the result of the judgment having the 

45 meaning between true to false value, the judgment value of ternary or higher value taking the value in the interval of [0, 
1] is used, which is a characteristic point. 

In thus composed image motion vector detecting device of the third embodiment, the method of extracting the cor- 
relation value with the motion vector of each region is same as in the first embodiment and is omitted herein. The reli- 
ability judgment information detecting part 8 functions as the time-course variation detection part 4 of the correlation 

so value and the correlation value detection part 3 in Fig. 13, and as the plural sets of information of reliability judgment 
,the minimum, mean .maximum of the correlation values , and the time differential value of the correlation values or the 
difference as the time-course variation of the correlation values, are obtained. In the first judging means 9-1 to 9-n, the 
reliability of each region is judged by the ternary or higher value taking the values in the interval [0, 1], with respect to 
each information from the plural sets of information about the correlation value entered from the reliability judgment 

55 information detecting part 8. The second judging means 10 determines one judgment value taking the values in the 
interval [0, 1] in each region by averaging or multiplying the plural judgment values obtained by the first judging means 
9-1 to 9-n. The judgment value converting means 1 1 converts the judgment value taking the values in the interval [0, 1] 
obtained by the second judging means 10 into a binary value of { 0, 1 } from a specific level boundary, and the result of 
the judgment is delivered to the motion vector determining means 6, together with the motion vector in each region. The 
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The swing correcting means 7, when receiving the image with large fluctuation as shown in 226, 227 in Fig. 5, 
moves the constitution of the image parallel from 229 to 228 in the direction 231 for correcting the motion, on the basis 
of the motion vector 230 entered from the motion vector determining means 6, with respect to the.input image signal of 
the (n-1)-th field 226, the n-th field 227, and enlarges or manipulates the constitution by interpolation or the like, and 
5 delivers the image signals 232, 233 in the size of one constitution small in swing. 

Thus, according to the embodiment, when judging the reliability of the motion vector in each detecting region, the 
ternary or higher judgment value in the interval [0, 1] is determined from each one of the plural sets of reliability judg- 
ment information in each detecting region, and by obtaining one ternary or higher judgment value from plural ternary or 
higher judgment values, the final binary judgment value in each detecting region is obtained. Consequently, even if one 
w of the plural sets of judgment information is fluctuating due to noise or the like, the reliability can be judged correctly, 
and if a moving object occupying the majority or whole ol the screen passes through the screen . malfunction of cor- 
recting in a wrong direction from the direction intended by the operator due to interruption of the correction is avoided. 
Meanwhile, in the third and fourth embodiments, the number of detecting regions in the screen may be four or other. 
The motion vector detecting device in a fifth embodiment of the present invention is described below. The configu- 
15 ration of the motion vector detecting device of the fifth embodiment is shown in Fig. 22. Those same as in Fig. 15 are 
identified with same reference numbers, and explanations are omitted. In Fig. 22, numerals 13-1, .... 13-n are judging 
means for judging the binary judgment value of {0, 1 } in every divided judgment interval , and 1 4 is judgement value con- 
verting means for converting from the plural judgment resuHs by the judging means into one binary judgment value of 
{0. 1}. In this embodiment, it is characteristic that the binary judgment value of {0, 1} is used in every divided judging 
20 interval as the judgment result having the meaning between true and false values. 

In thus composed image motion vector detecting device of the fifth embodiment, the method of obtaining the 
motion vector of each region and the reliability judgment information is same as in the third embodiment and is not 
explained herein, in the judging means 13-1 to 13-n, the reliability of each region is judged for each information from 
the plural sets of information of the correlation value entered from the reliability judgment information detecting part 8. 
25 !n each judgment interval divided with respect to each judgment information in every region, the binary judgment of {0, 
1} is effected. The judgment value converting means 14 converts from the binary judgment value of {0, 1} in the plural 
judgment intervals obtained in each region into one binary judgment value of {0. 1}, and the result of judgment is deliv- 
ered to the motion vector judging means 6 together with the motion vector in each region. The motion vector judging 
means 6 delivers the intermediate value of the motion vector from the region judged to be reliable, as the motion vector 

30 of the screen. . . 

The method of deduction of the judgment result in each region different from the motion vector detecting device in 
the third embodiment is further described below. For the sake of simplicity of explanation, the number of pieces of reli- 
ability judgment information is two, but this s not limitative. Fig. 23 shows the detailed configuration of the judging means 
1 3 and judgment value converting means 1 4. In Fig. 23, numerals 1 3-1 (a) to 1 3-1 (d), and 1 3-2(a) to 1 3-2(d) are judging 

35 circuits for delivering the binary judgment value of {0, 1} in every judging interval, and the judgment value converting . 
circuit 14 is composed of AND and OR circuits. The output signals of the judging circuits 13-1 (a) to 13-1(d) are shown 
in Rg. 24 (a) to (d), and it is same in 13-2(a) to 13-2(d). That is. 13-1 (a) is 1 when the judgment information 1 is from p 
to q, 13-1{b) is when q to r, 13-1(c) is when r to s, and 13-1(d) is when higher than s, and otherwise 0. It is false when 
all of 13-1 (a) to 13- 1(d) are 0, and true when 1 3-1 (d) is 1 , and intermediate in the meaning from true to false when any 

40 one of 13-1 (a), (b), (c) is 1 , which shows the result of judgment. In this case, in order to have the meaning between true 
and false by using a binary value of 0 or 1 , the judgment region is divided into five intervals as less than p, p to q, q to 
r, r to s, and more than s as shown in Rg. 24, as for the ambiguous judgment interval W1 shown in Rg. 6. It means the 
same effect as using the five discrete values for judgment from 0 to 1, by the function of the judgment values for the 
whole region of judgment as shown in Fig. 18 in the third embodiment. In other words. 13-1 (a). 13-2(a) corresponds to 

45 0.25, 13-1{b), 13-2(b)to0.5. 13-1(c), 13-2(c) to 0.75. and 13-1 (d), 13-2(d) to 1. thus corresponding to the intermediate 
judgment values from 0 to 1 . Suppose 1 3-1 (b) to be 1 from the judgment information 1 . At this time, the final judgment 
result of the judgment value converting means 14 becomes 1 when the judgment result by the judgment information 2 
is 1 in 13-2(c) or 1 in 13-2(d). In other words, seeing that 13-1(b) corresponds to 0.5, 13-2(c) to 0.75, 13-2(d) to 1. it 
means that the final judgment result is 1 when the mean of the intermediate judgment value from 0 to 1 by two sets of 

so judgment information is greater than 0.5. This is the same result as shown in the third embodiment. 

Thus, the method of obtaining the judgment result in the embodiment is to determine the binary value of {0. 1} in 
every divided judgment interval corresponding to every one of plurality of reliability judgment information in every 
detecting region, and to obtain one binary judgment value of {0 or 1}1 from the binary judgment values of {0, 1} of plural 
judgment intervals, thereby leading out the final binary judgment value in each detecting region. Accordingly, express- 

55 ing the ambiguity of the threshold level in the judgment of binary value of {0, 1 } in every divided judgment interval, and 
if one of the plural judgment results be ambiguous, correct judgment is achieved. As far as the function is the same as 
mentioned in this embodiment, other method may be also employed. 

Rg. 25 is a block diagram of a swing correcting device in a sixth embodiment of the present invention. Those same 
as shown in Fig, 14. Fig. 22 are identified with same reference numbers, and explanations are omitted. 
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What is different from the motion vector detecting device in the fifth embodiment is the method of determining the 
motion vector of the screen which is described in detail below. 

In the vector detecting circuit 2 of each detecting region, preliminarily, as indicated by 209, 210, 211, 212 in Fig. 2, 
four vector detecting regions are determined in the screen, and in each detecting region between two fields, the differ- 

5 ence E|AL|(i,j) of the signal of the position deviated by the quantity (i,D:imin < i< imax, jmin < j < jmax in a specified range 
is determined as the correlation value, and the deviation (i'j 1 ) for giving the minimum value of this value is detected, and 
it is obtained as the vector Vi (i = 1 to 4) in each region. 

In the motion vector determining means 16, the motion vector V of the screen is determined from the vector Vi of 
each region and the binary judgment value of {0, 1) in each divided judgment interval. Fig. 29 shows a detailed consti- 

10 tution of the motion vector determining means .16. The motion vector determining means 16 comprises weighting 
means 1 7-1 to 1 7-8 of the vectors in each region and an adder. A detailed configuration of the vector weighting means 
17 is shown in Fig, 30. The vector weighting means 17 consists of attenuator 18 and switching element. The weighting 
means 17 receives, besides the vector Vi in each region, the binary judgment value of{0, 1} in each judgment divided 
interval from the judging means 13, and the switching element receiving the judgment value of 1 is turned on. In con- 

is sequence, the damping factor of the attenuator connected to the switching element which is turned on is multiplied by . 
the vector V, and the product is delivered from the weighting means 1 7. The vector Vi in each region is multiplied by the 
damping factor in the above method by the two weighting means as shown in Fig. 29, and added up in the adder. The 
output after addition becomes the vector Vi weighted by the sum of the damping factors multiplied by the two weighting 
means. Here, the weight of the vector Vi after addition corresponds to the weight coefficient ai in the seventh embodi- 

20 merit, and in this embodiment, too, the sum of the damping factors is expressed as ai. The vector Vi weighted with ai 
in each detecting region is summed up in four regions by the adder, and the final motion vector V of the screen is 
obtained. This is expressed in the following formula. 

V = a1*V1+a2*V2+a3*V3+a4*V4 

25 

This equation is exactly same as the formula of determining the vector V in the seventh embodiment. 

Thus, in the motion vector detecting device of the embodiment, the binary judgment value of {0, 1 } is determined in 
each judgment interval divided corresponding to each one of plural sets of reliability judgment information in each 
detecting region, and the weight coefficient of the vector in each detecting region is determined by the damping factor 
30 corresponding to the interval in which the judgment is 1 . From this weight factor and the vector in each detecting region, 
the motion vector of the screen is finally determined. As a result, the ambiguity ol the threshold level is expressed by 
the binary judgment of 0, 1 in each divided judgment interval, and the motion vector may be determined by reducing the 
weight in the region with ambiguous result of judgment 

Fig. 31 is a block diagram of a swing correcting device in a tenth embodiment of the present invention. Those same 
35 as in Fig. 14, Fig. 28 are identified with same reference numbers and are not explained herein. 

The blocks of the image motion vector detecting device of thus composed tenth embodiment are same in operation 
as the ninth embodiment, and its explanation is omitted. 

A swing correcting means 7, when receiving an image with large fluctuation as indicated by 226, 227 in Fig. 5, 
moves the screen of the image parallel from 229 to 228 in a direction 231 of correcting the motion on the basis of the 
40 motion vector 230 entered from the motion vector determining means 1 6, with respect to input image signals in the (n- 
1)-th field 226, the n-field 227, and enlarges or manipulates the screen by interpolation or the like, thereby delivering 
image signals 232, 233 in a size of one screen small in fluctuation. 

Thus, in this embodiment, in each detecting region, the binary judgment value of {1 , 2) is obtained in each judgment 
interval divided with respect to each one of plural sets of reliability judgment information, and the weight coefficient of 
45 the vector in each detecting region is determined by the damping factor corresponding to the interval in which the judg- 
ment is 1. From this weight coefficient and the vector in each detecting region, the motion vector of the screen is finally 
determined. As a result, by the binary judgment of {0, 1} in each divided judgment interval, the ambiguity of the thresh- 
old level is expressed, and the motion vector may be determined by reducing the weight in the region with ambiguous 
result of judgment and , even H a large object occupying the majority or whole of the screen passes the screen, the mal- 
so function of correction of the screen in wrong direction from the direction intended by the operator due to interruption of 
the correction is avoided. Meanwhile, in the ninth and tenth embodiments, the number of detecting regions in the screen 
is four, but it may be also a different number. 

The motion vector detecting device in an eleventh embodiment of the present invention is described below. The 
configuration of the motion vector detecting device of the eleventh embodiment is shown in Fig. 32. This embodiment 
55 is based on the conventional motion vector detecting device, and those same as in the prior art in Fig. 1 are identified 
with same reference numbers, and explanations are omitted. What is different from the prior art is that the motion vector 
correlation value calculating means 19 and judgment condition varying means 20 are provided as shown in Fig. 32. 

In thus composed image motion vector detecting device of the eleventh embodiment, preliminarily as indicated by 
209. 210, 211, 222 in Fig. 2, four vector detecting regions are determined in the screen, and the vector Vi (i = 1 to 4) of 
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below The configuration of the motion vector detecting device of the seventeenth embodiment is shown in Flo, 39. This 
embodiment is based on the motion vector detecting device of the third embodiment, and those same as in the config- 
uration of the third embodiment in Fig. 15 are identified with same reterence numbers and the explanations are omitted. 
What is different from the third embodiment is that the motion vector correlation value calculating means 19 and judg- 
ment condition varying means 20 are disposed as shown in Fig. 39. 

In thus composed image motion vector detecting means of the seventeenth embodiment, the operation of the 
motion vector correlation value calculating means 19 is same as explained in the eleventh embodiment The judgment 
condition varying means 20 changes the conversion level of the judgment value varying means 1 1 for comparing the 
specified conversion level with the ternary or higher judgment value having the value in interval [0. 1] obtained by the 
second judging means 10 . so that the judgment condition may be made less severe when the correlation value C of 
the motion vector is less than the specified value. Or it may be also possible to prepare plural judgement value convert- 
ing means differing in the conversion level, and change over the judgment value converting means depending on the 
correlation value C. The number of changes of the judgment conditions by the correlation value is not limited to two 

al ° n Rg. 40 is a block diagram of a swing correction device in an eighteenth embodiment of the invention. Those same 
as in Fig 14 Rg 30 are identified with same reference numbers, and are not explained herein. 

In thus composed image motion vector detecting device of the eighteenth embodiment, the operation is same as 
in the seventeenth embodiment, and its explanation is omitted. The operation of the swing correcting means is also 
same as explained in the second and fourth embodiments, and its explanation is omitted. In the seventeenth and eight- 
eenth embodiments, the number of detecting regions in the screen may be either four or other. 

Next the motion vector detecting device of a nineteenth embodiment of the invention is explained. The configura- 
tion of the motion vector detecting device of the nineteenth embodiment is shown in Fig. 41 . This embodiment is I based 
on the motion vector detecting device of the fifth embodiment, and the parts same as in the configuration of the .fflh 
embodiment in Fig. 22 are identified with same reference numbers, and their explanation is omitted. What is different 
from the fifth embodiment is that the correlation value calculating means 19 of motion vector and judgment condition 

varying means 20 are disposed as shown in Fig. 41. ««i ,= 

In the image motion vector detecting device of thus composed nineteenth embodiment, the operation of the corre- 
lation value calculating means 19 of motion vector is same as explained in the eleventh embodiment The judgment 
condition varying means 20 changes to make the judgment condition of the judging means 1 3 less severe when tfie cor- 
relation vector C of the motion vector is less than s specific value. That is. the judgment intervals from p to s shown in 
Fig 24 are moved to the less severe direction of the judging condition, it may be also possible to prepare plural judging 
means differing in the judging intervals, and change over the judging means depending on the correlation value C. The 
number of changes of judging condition by the correlation value is not limited to two. 

Fig. 42 is a block diagram of a swing correcting device in a twentieth embodiment of the invention. Those same as 
in Fig 14 Rg 41 are identified with same reference numbers, and the explanation is omitted. 

The blocks of the image motion vector detecting device of thus composed twentieth embodiment are same in oper- 
ation as the nineteenth embodiment, and its explanation is omitted. The operation of the swing correcting means is ateo 
same as in the second and fourth embodiments, and the explanation is omitted. In the nineteenth and twentieth embod- 
iments, the number of detecting regions in the screen may be either four or other. 

Next the motion vector detecting device in a twenty-first embodiment of the invention is described below. The con- 
figuration of the motion vector detecting device of the twenty-first embodiment is shown in Fig. 43. This embodiment is 
based on the motion vector detecting device of the seventh embodiment of the invention, and those same as in the con- 
figuration of the seventh embodiment in Fig. 26 are identified with same reference numbers, and are ™t explained 
herein. What is different from the seventh embodiment is that correlation value calculating means 19 of motion vector 
and judgment condition varying means 20 are disposed as shown in Rg. 43. 

In the image motion vector detecting device of thus composed twenty-first embodiment the operation of the corre- 
lation value calculating means 19 of motion vector is same as explained in the eleventh embodiment. The judgment 
condition varying means 20 changes the function of the judgment value to make the judgment condition less severe as 
shown in Fig 37 when the correlation value C of the motion vector is less than a specific value. In Fig. 37. meanwhile, 
the function of the judgment value is moved parallel, but it is not limitative, and the shape of the function may be varied. 

Next Rg 44 is a block diagram of a swing correcting device in a twenty-second embodiment of the invention. 
Those same as in Fig 1 4 Fig. 43 are identified with same reference numbers, and are not explained herein. 

The blocks of the image motion vector detecting device of thus composed twenty-second embodiment are same in 
operation as the twenty-first embodiment, and its explanation is omitted. The operation of the swing correcting means 
; is also same as explained in the eighth embodiment, and is no described herein. In the twenty-first and twenty-second 
embodiments, the number of detecting regions in the screen may be either four or other. 

Next the motion vector device of a twenty-third embodiment of the invention is described. The configuration of the 
motion vector detecting device of the twenty-third' embodiment is shown in Fig. 45. This embodiment is based on the 
motion vector detecting device of the ninth embodiment of the invention, and those same as in the configuration of the 
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detecting region, as for each detecting ^ ^.^^£S5J2 «£ eolation value, and the deviation 
ated by a specified quantity (..,):im,n < .< max J^n <| < Bd ^™ r a each regjon . In the correlation value 
( r.n for giving the minimum value is m Sum of co^" values are obtained in each detecting 

detecting parts 23a to 23d. ^ "^^.^^T^J^, J of region is judged by the correlation 
region. In the vector reliability judging 0 , judgment and the motion vector of 

value entered from the correlate value detecting ^^SAart the resu t J g rmjnj ^ djfference 

each region are delivered to the motion vector d «^^ m "^„^^ is deter mine. while the difference 
of the correlation values, the difference «£^*£^£ZZZI£i £ method for determining me 
between the upper and lower regions on the , ^ , f ^^ J™ ^J le(l re g ions of the screen , either all four 
difference of correlation values in the upper and lower regions or the right and «* differences of 

regions or only two regions in Fig. 2 may be used. Meanwhile ^^^^ motion vector deter- 

the upper and lower regions, and right and left regions may be a specific value, deter^ 

mining means 27, when the difference of ^??™SSS!3^^ ™ M * M 

mines the intermediate valueof thevectorsof the t ^^^^^SZ numb er) as the motion vector of the 

- ,h6 S~oS^^^^ 

is not determined only by the result of judgment of detecting region. 

* UPP ^=mSv=Sr^ 

between regions when a moving object enters the ^^^^^m^. *>' id line in < C > is 
Fig. 48, (a) denotes the motic. vector of the » ' B *Zl£Z^jE<& region, and (d) denotes 

the correlation value of the left region the br °^J n j^ ^ increases as the moving objects 

so the difference of correlation values of the right and '^9'on^The u mwjng 

invades the detecting region, and decreases as the antra region s .occupied by h e m o* g M means 
object leaves the detecting region, the correlation value begins ^^"^^^J^noii in the prior art. the 
24 the reliability is judged on the basis of the correlation l 9 ^^^^^ J JTmeans 25 for deter- 
moving object is often misjudged if judged in each 7?'°™ d J^^^^ of invasion of the 

55 mining the difference of the right and left regions o the ^ e '^ 
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screen is set at 0 for a specific period. As a result, in the reliability determining means in each region, even if the moving 
object is misjudged, the invasion of the moving object is detected by the difference signal between regions. In this 
embodiment, meanwhile, the number of detecting regions in the screen is four, but it is not limitative. 

Next, the motion vector detecting device of a twenty-sixth embodiment is described below. The configuration of the 

5 motion vector detecting device ol the twenty-sixth embodiment is shown in Fig. 49. Those same as in Fig. 47 are iden- 
tified with same reference numbers, and are not explained. In Fig. 49, numerals 28a to 28d are vector reliability judging 
means, and 29 is motion vector determining means. 

in the image motion vector detecting device of thus composed twenty-sixth embodiment, the method of leading out 
the vector of each region and correlation values and the method of determining the difference of correlation values 

10 between regions are same as in the above-mentioned embodiment, and the explanation is omitted. In the vector relia- 
bility judging means 28a to 28d, the reliability of each region is judged by the correlation values entered from the corre- 
lation value detecting parts 23a to 23d, and besides by the difference of correlation values between regions entered 
from the means 25 for determining the difference of correlation values between right and left regions and from the 
means 26 for determining the difference between the upper and lower regions, and the result of this judgment and the 

is motion vector of each region are delivered to the motion vector determining means 29. The motion vector determining 
means 29 determines the intermediate value of the vectors of the regions judged to be reliable (the value positioned in 
the middle when arranged in the size order, or the mean of two central values in the case of an even number) as the 
motion vector of the entire screen, and delivers. 

What is different from the conventional motion vector detecting device is that the reliability of each detecting region 

20 is not judged by the correlation value only in each detecting region, but is judged by the difference in the correlation val- 
ues between detecting regions as well as the correlation value in each detecting region. This point is explained in fur- 
ther detail below. The following explanation relates only to the case of upper and lower regions, and the case of right 
and left regions is the same and is omitted herein. 

When a moving object enters the screen from left to right, the operation is described referring to Fig. 48. In Fig. 1 8, 

25 (a) to (d) are as explained already in the twenty-fifth embodiment. Now, the difference signal of the right and left regions 
of the correlation values in Fig. 48 (d) is obtained by subtracting the correlation value of the right region from the corre- 
lation value of the left region. As shown in this diagram, as the moving object passes on the screen, the difference signal 
of the correlation value fluctuates largely to both positive and negative sides. In the reliability judging means 28a, 28b 
(the reliability judging means of the left region), in addition to the correlation values, the difference signal of the correla- 

30 tion values is compared with the positive specified level, and the reliability is judged, while in the reliability judging 
means 28c, 28d (the reliability judging means of the right region), in addition to the correlation values, the difference 
signal of the correlation values is compared with the negative specified level, and the reliability is judged. In the motion 
vector determining means 29, as mentioned above, the intermediate value of the vectors of the regions judged to be 
reliable (the value positioned in the middle when arranged in the size order, or the mean of two central values in the 

35 case of an even number) is determined and delivered as the motion vector of the entire screen. 

In this embodiment, thus, by using the correlation values of the regions and the difference of correlation values 
between regions in the reliability judgment in each region, the judging performance of the reliability for a moving object 
is enhanced. In this embodiment, meanwhile, the number of detecting regions in the screen is four, but it may be also 
other number. 

40 Next the motion vector detecting device of a twenty-seventh embodiment of the invention is descried below. The 
configuration of the motion vector detecting device in the twenty-seventh embodiment is shown in Fig. 50. Those same 
as in Fig. 47 are identified with same reference numbers and are not explained herein. In Fig. 50, numerals 30a to 30d 
are time variation detecting parts of correlation values, 31a to 31 d are vector reliability judging means, 32, 33 are means 
for determining the difference of the time variation values of correlation values, and 34 is motion vector determining 

45 means. 

In the image motion vector determining device of thus composed twenty-seventh embodiment, the method of 
deducing the vectors of each region and correlation values is same as in the twenty-fifth embodiment, and its explana- 
tion is omitted. In the time variation detecting part 30 of correlation values, the time differential value or the difference 
value after averaging the correlation values is determined. The vector reliability judging circuit means 31 judges the reli- 

50 ability of each region from the correlation values entered from the correlation value detecting part 23, and from the time 
variation values of correlation values entered from the time variation detecting part 30 of correlation values, and delivers 
the result of the judgment to the motion vector determining means 34, together with the motion vector of each region. 
In the means 32 for determining the difference of the time variation value of correlation values, the difference as to right 
and left regions of the screen is determined, while the difference as to upper and lower regions of the screen is deter- 

55 mined in the means 33. As the method of determining the difference of the time variation values of the correlation of the 
upper and lower or right and left regions of the screen, either all four regions, in Fig. 2 or only two regions may be used. 
Or when two diagonal regions and used, the differences of the upper and lower, and right and left regions may be 
detected at the same time. The motion vector determining means 34 determines the intermediate value of the vectors 
of the regions judged to be reliable (the value positioned in the middle when arranged in the size order, or the mean of 
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taking the time on the axis of abscissas. In Fig. si, aj a enou» u variation 
vector of the right region, (c) is the time ™ab°n ^ of correction values 

valueofcorrelationvalueoftherigrtreg^^ the negative portion 

between.the right and let. regions, when determining the difference! 

of the time variation va.ue of the correlation ^^J^^^^^^^ 
31 . the reliability is judged in ^ ^ fe ^ h 

of judgment of reliability in each region. In the vecto determ ™ng T^J^^^e* a spectfic value, the 
signal of the right and left regions a 

motion vector of the entire screen .s set at 0 for a specrf c penod JW^ ^ jpva . 
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the difference signal of the time variation value of the correlation value fluctuates largely to the positive and negative 
side. Accordingly, in the reliability judging means 35a, 35b (the reliability judging means of the left region), in addition to 
the correlation value and the time variation value of the correlation value, the difference signal of the time variation value 
of the correlation value is compared with the positive specified level, and the reliability is judged, and in the reliability 

5 judging means 35c, 35d (the reliability judging means of the right region), in addition to the correlation value and the 
time variation value of correlation value, the difference signal of the time variation value of correlation value is compared 
with the negative specified level to judge the reliability. In the motion vector determining means 36, as mentioned above, 
the intermediate value of the vector in each region judged to be reliable (the value positioned in the middle when 
arranged in the size order, or the mean of two central values in the case of an even number) is determined and delivered 

10 as the motion vector of the entire screen. 

In this embodiment, thus, in judgment or reliability of each region, in addition to the correlation value of the each 
region and time variation value of correlation value, by using the time variation value of the correlation values between 
regions, the judging performance of reliability of the moving object may be enhanced. In this embodiment, meanwhile, 
the number of detecting regions in the screen is four, but it is not limitative. Besides, when determining the difference 

is between regions, the negative portion of the time variation value of correlation values individual regions was supposed 
to be 0, but it is not always necessary to set at 0. 

Next Fig. 53 is a block diagram of a swing correcting device in a twenty-ninth embodiment of the invention. In Fig. 
53, numeral 37 is a motion vector detecting device, and 38 is swing correcting means. Incidentally, the structure of the 
motion vector detecting device 37 may be any one of those explained in the twenty-fifth to twenty-eighth embodiments. 

20 In thus composed image motion vector detecting device of the twenty-ninth embodiment, the operation is same as 
in the twenty-fifth to twenty- eighth embodiments, and its explanation is omitted. 

The swing correcting means 38, when receiving an image with a large fluctuation as indicated by 226, 227 in Fig. 
5, moves the screen of the image parallel from 229 to 228 in the direction 231 of correcting the movement, on the basis 
of the motion vector 230 entered from the motion vector detecting device 37, with respect to the input image signals of 

25 the (n-1)-th field 226 and the n-th field 227, and manipulates the screen by enlarging or the like by interpolation or the 
like, thereby delivering image signal 232, 233 in the size of one screen small in fluctuation. 

Thus, according to this embodiment, whether the moving object has invaded the region or not is judged, in each 
detecting region, by the correlation value and the time variation value of correlation value, and the moving object is 
detected by finding the correlation value between regions and the difference of time variation value of correlation val- 

30 ues, and the motion vector of the entire screen is determined. On the basis of thus determined motion vector of the 
entire screen, the image signal memory means and CCD reading position are controlled, thereby correcting the swing 
of the screen. As a result, even if a moving object occupying the majority or whole of the screen passes through the 
screen, malfunction of correcting the screen in a wrong direction from the direction intended by the operator due to 
interruption of correction is avoided. 

35 Next the motion vector detecting device of a thirtieth embodiment of the invention is explained. The configuration 
of the motion vector detecting device of the thirtieth embodiment is shown in Fig. 54. Numeral 301 is an image signal 
input terminal, 302a to 302c are vector detecting circuits of each detecting region, 303a to 303c are correlation value 
detecting parts, 304a to 304c are mean calculating parts, 305a to 305c are time variation value detecting parts, 306 to 
308 are means for determining the difference of the time variation values between regions, 309 is moving object detect- 

40 ing means, 310 is reliability judging means, and 31 1 is motion vector determining means. 

In the image motion vector detecting device of thus composed thirtieth embodiment, first, an image signal consecu- 
tive in time for at least two fields is entered into the input terminal 301 . In the screen, three vector detecting regions 
are preliminarily determined as indicated by 312, 313, 314 in Fig. 55, and in vector detecting circuits 302a to 302c of 
the detecting regions, the difference £|Al_|(i,j) of the signal at the position deviated by a specified quantity (i,j):imin < i< 

45 imax, jmin < j < jmax is determined as the correlation value in each detecting region between two fields, and the devia- 
tion (i\j') for giving the minimum value of the value is detected, and it is regarded as the vector of each region. In the 
correlation detecting parts 303a to 303c, the minimum, mean and maximum of correlation values are determined in 
each detecting region. In the mean calculating parts 304a to 304c, the time mean is determined for the minimum of the 
correlation values. In the time variation value detecting parts 305a to 305c, the time differential value or the difference 

so value after averaging the minimum values of the correlation values determined. In the means 306 for determining the 
difference of the time variation values between regions, the difference between the left region and right region of the 
screen is determined, while the difference between the middle region and right region of the screen is determined in 
307, and the difference between the middle region and left region of the screen is determined in 308. Numeral 309 is 
for detecting the moving object on the basis of the difference signal between regions obtained by 306 to 308. Tne vector 

55 reliability judging means 310 judges the reliability in each region from the correlation value entered from the correlation 
value detecting part 303 and the moving object detecting signal from the moving object detecting means 309, and the 
result of judgment and the motion vector of each region are delivered to the vector determining means 311. The motion 
vector determining means 31 1 determines the intermediate value of the vectors of each region judged to be reliable by 
the reliability judging means (the value positioned in the middle when arranged in the size order, or the mean of two 
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central values in the case of an ever, number) as the motor, vector of the entire screen and defers. 

Here what is different from the conventional motion vector detecting device is that the mobon vector of the entire 
=^ Jn ! nnt dl erm^ed only by the reliability judgment result in each detecting region, but by the drtference in the tame 
^SSZ^^^i ^ions as we.l as the resuH of judgment o. re.iabi.ity in each detecting 

r ^™^^^ «*» - right detecting regions, output si fl na.s of ft. tame variation 

vJ3JZ£^S£2**° of the minimum value of correlation values, and the difference of the M«M 
values be^enSons when a moving object enters the screen from left to right, taking the tame on the axs of ataos- 
sa Tn fTs6 (a) ^denotes me motion vector of the left region, (b) is the motion vector of the middle reg,on (c) is he 
nlSion vlctoTof the right region, (d) is the time variation value of the left region, (e) is the tame variation valuertthe 
ThZ rfnY™ 7« is the time variation value of the right region, (g) is the difference in the time vanation value between 
T^^£!iS!^aSSSi the difference in time variation value between the middle and right regions. 
^.iSSSS^S regions, however, the negatave portion of the time variation value of the co, 
When determining line °'™ r °" . t b Q |n ft means 306 for determining the difference of time variation 

cific time, it is judged that the moving object has invaded the screen. 

Condition 1: difference signal 315 > a and f51 > difference signal 316 > p2 
Condition 2: difference signal 31 6 > y 

This is the explanation of the case of the moving object entering the screen from left to right, but if entering from 
JTJZ th« SStSOm be similarly detected. In this case, instead of the difference of the time variation value 
bCe^SSS 

"'In the reliability judging means 310. the reliability is judged by using the <*^.^^^£*£Z. 
reliable is judged on the basis of the moving object detecting signal from the moving ob,ect detertng means 309. Con 

ce« 

ntaueuse^ Se forego ng embodimerrts of the invention may be emo^ 

on ^s orthe mwing object detecting signal, when the moving object is detected by the moving object de ecting 
means^9 the reliability * negated in all regions. The motion vector determining means 31 1 , dete rmines tht J-™* 
SevieofL 

35 ^s^coS^^ 

the chelation value in each region fails to detect the moving object, the invasion of the mcvng ob ect may be detected 
bTthedSence ^between region, Therefore, the motion vector due to camera shake or the like may be 

M ^"^L embodiment the number of detecting regions in the screen is three, but this is not limitative. Furthermore in 
this 2SSS Z i^Tt^ the n*£ is determined in the lateral direction in order to detect the = g 

between the regions may be determined in the vertical direction. An example w.th f ,ve detecting regions .s g^en ,n Bg. 

45 in the embodiment, when determining the difference between regions, the negative portion of the time variation 
value of correlation values in each region is supposed to be 0, but it is not always reqmred to be 0. 

Next L motion vector detecting device of a thirty-first embodiment of the invention .s descned The ^"ration 
of theTotion dSng device of the thirty-first embodiment is shown in Rg. 58. Those same as ,n F,g. 54 are >dentif ed 
sam 7efefenc numbers, and the Explanation is omitted. What is different from the thirtieth embodimert i* ^ the 
mrvinl objS detecting means 317 and the reliability judging means 318. and only these two ports are descnbed 

^tn example of detecting a moving object entering the screen from the left side is describe below. In the screen. 
three^eS*^ 

no the difference between the left and right regions of the time variation value, and ill the means 307 for determining 
Z 2 e ;en^ronhe^dd"e and right regions, due to the time difference of the moving object entering each region, dif- 
^S^^TS^SSL a 9 s shown in Rg. 56 (g), (h), and are sen, to the moving object d***^ 
3 r?n thfrnoling object detecting means 317. after the difference signals 315, 316 satisfy the followmg condition ^ 
vleniecoS 
has invaded the screen. 
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Condition 1 : 
Condition 2: 
Condition 3: 
Condition 4: 



difference signal 315 > a and p1 > difference signal 316 > p2 

difference signal 316 > y1 

difference signal 31 6 > y2 

difference signal 31 6 > *ft however, yl < 72 < 73 



5 



In the reliability judging means 318. while judging the reliability by using the correlation value of each region, the 
reliability is judged on the basis of the moving object detecting signal from the moving object detecting means 317. As 
the technique for judging the reliability by using the correlation value, either the conventional method or the technique 
used in the other embodiments ol the invention may be employed. As the technique for judging the reliability on the 

jo basis of the moving object detecting signal, when the condition 2 is satisfied by the moving object detecting means 31 7, 
the vector detecting region 312 in Fig. 55 is judged to be not reliable, and when the condition 3 is satisfied, the vector 
detecting region 313 is judged to be not reliable, and when the condition 4 is satisfied, the vector detecting region 314 
is judged to be not reliable. However, when the condition 4 is not satisfied within a specific time, the.vector detecting 
regions judged to be not reliable in the condition 2 and condition 3 are judged to be reliable. Thus, in this embodiment, 

15 it is characteristic that the reliability is negated sequentially from the region invaded by the moving object. The motion 
vector determining means 31 1 determines the intermediate value of the vector in each region judged to be reliable by 
the reliability judging means 318 as mentioned above as the motion vector of the entire screen, and delivers it. 

In such constitution of the embodiment of the invention as described above, even if the moving object cannot be 
detected by the reliability judging means by the correlation value in each region, the invasion of the moving object can 

20 be detected by the difference signal between regions. Therefore, the motion vector due to camera shake or the like can 
be corrected detected. 

In the embodiment, the number of detecting regions in screen is three, but it is not limitative. Furthermore, in this 
embodiment, the difference between regions is determined in the lateral direction to detect the object moving in the lat- 
eral direction, but the difference between regions may be determined in the vertical direction to detect an object moving 

25 in the vertical direction. Incidentally, the condition 4 of the moving object detecting means 31 7 of the embodiment may 
not be absolutely necessary, and without it, the vector detecting region 313 in Fig. 55 is judged to be not reliable when 
either the condition 2 or condition 3 is satisfied. In the embodiment, when determining the difference between regions, 
the negative portion of the time variation value of correlation value in each region is supposed to be 0, but it is not 
always necessary to suppose to be 0. 

30 Next the motion vector detecting device of a thirty-second embodiment of the invention is described below. The 
configuration of the motion vector detecting device of the thirty-second embodiment is shown in Fig. 59. Those same 
as in Fig. 54 are identified with same numbers, and are not explained herein. In Fig. 59, numeral 319 is reliability judging . 
means, and 320 is vector determining means. 

As an example of invasion of a moving object from the left side of the screen, only different points from the thirtieth 

35 embodiment are explained below. In the reliability judging means 319. the reliability is judged in each region by using 
the correlation value of each region. The technique of judging the reliability by using the correlation value may be either 
the conventional technique or the technique used in the other embodiments of the invention. The motion vector deter- 
mining means 320, as mentioned above, determines the intermediate value of the vector of each region judged to be 
reliable by the reliability judging means 31 9 as the motion vector of the entire screen and delivers it, but when the mov- 

40 ing object is detected by the moving object detecting means 309, regardless of the result of the reliability judging means 
319, the motion vector of the entire screen is assumed to be 0 for a specific period. As a result, even if the moving object 
is misjudged by the reliability judging means in each region, the invasion of the moving object is determined by the mov- 
ing object detecting means, and the motion vector due to camera shake or the like may be correctly detected. 

Next is explained the motion vector detecting device of a thirty-third embodiment of the invention. The configuration 

45 of the motion vector detecting device of the thirty-third embodiment is shown in Fig. 60. Those same as in Fig. 54 are 
identified with same reference numbers, and the explanation is omitted. In Fig. 60, numeral 321 is reliability judging 
means. 

As an example of invasion of a moving object from the left side of the screen, only different points from the thirtieth 
embodiment are described below. In the reliability judging means 321, the reliability is judged in each region by using 

so the correlation value of each region, and when the moving object invading from the left is detected by the moving object 
detecting means 309, the vector detecting region 312 in Fig. 55 is judged to be not reliable. The technique of judging 
the reliability by using the correlation value may be either the conventional method or the technique used in the other 
embodiments of the invention. Furthermore, when detecting the moving object by the moving object detecting means 
309, the reliability judging condition of the reliability judging means 321 is varied so as to raise the detecting sensitivity 

55 of the moving object. As a result, even if the moving object is misjudged in the reliability judging condition of the ordinary 
reliability judging means, the invasion of the moving object is detected by the moving object detecting means, and by 
varying the reliability judgment condition, the motion vector due to camera shake can be corrected detected. 

The motion vector detecting device of a thirty-fourth embodiment of the invention is described below. The configu- 
ration of the motion vector detecting device of the thirty-fourth embodiment up to the moving object detecting means is 
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Sr* signa. from the moving object detecting means 323 shown ,nRj 6t , * -jay P ^ ^ 
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means between regions, and 323 is moving object he value of the difference of the motion 

,n the vector difference detecting means ^^^^jJS^ vector of the right region (314 in 

vector of the left region (312 in Fig. *«£^^^t*£l 3«* «*"*■ «"""» -d " 

Fig. 55) is determine. In the mov.ng ob]ect detectmg means 323. me r^ g j ^ ^ d)i(erence 

period, it is judged that the moving object has mvaded the screen. 

m* by «MWp fx*™"* »"'» ^£5^211? to . m«ng cbjsa k 1»* 

swing correcting means 208 in F.g. 1 . comoosed thirty-fifth embodiment, first, an image signal con- 

in the image motion vec*c,dete^^ 
secutive in time tor at least two fields .s entered by 209 . 210 211, 212 in Fig. 2, and asfor each 

four vector detecting regions are determined ,n ^^^^^^^^^a^Mc^ 
30 detectingregionbetweentwofields,^ 
( U yWn<i<inr«x,jmin<^m^ 

value of the value is detected, and rt .staken as the s mcto ^ ^ ^ judg , ng rne 43. 

regions: k = 1 . 2, 3. 4). The mot,on vector ««.«* in each detecting region is still or not. and the result of 
The still state judging means fudges « the motor « > J, ^ TOtion vector 0 f each detertng. 
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where 

Sk: judgment result {1 if true, 0 if false) (k-1 ,2,3,4) 
t: time 
C: constant 

Vk: motion vector of each detecting region (k=1 ,2.3.4) 
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tor rel.abi.rty judging means 44 judges £ ^^%^2^y^^^«^^^^ 
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judgment of the reliability judging means and motion vectors of detecting regions are delivered to the vector determining 
means. The motion vector determining means 46 determines the motion vector of the entire screen from the judgment 
result of each detecting region and the motion vector of each detecting region. 

This operation is shown in a flow chart in Fig. 63. Starting from step 101, step 102 corresponds to 42 in Fig. 62, 

s step 103 to 44 in Fig. 62, step 104 to 43 in Fig. 62 steps 105, 106, 107 to 45 in Fig. 62, step 108 to 45 in Fig. 62, and 
the motion vector is delivered at step 109. 

Fig. 64, Fig. 65 are diagrams showing practical examples. Intervals (a) to (e) in Fig. 65 correspond to (a) to (e) in 
Fig. 64. Fig. 64 shows the correlation value between the screen and each detecting region when a moving object 52 
enters the screen, and Fig. 65 shows the motion vector 59 detected at this time. In Rg. 65, numeral 60 denotes a motion 

to vector of camera, and it is 0 vector. Numeral 56 is a motion vector of a moving object, and it is nearly constant. Numer- 
als 57, 58 are vectors detected in detecting regions 48, 49, and depending of the manner of relation between the mov- 
ing object and the detecting regions, the vector 56 comprising the motion of the moving object, and other wrong vectors 
are detected. Numeral 59 is a motion vector delivered from the motion vector determining means 46. 

First, as shown in Rg. 64 (a), when both the background and the moving object 52 are present in a certain detecting 

is region 48, this detecting region is judged to be not reliable depending on the state of the correlation value 53 as shown 
in Fig. 64 (a). It is desired that the detecting region 49 be judged to be reliable, but depending on the manner of repro- 
duction(image signal) of the background, the reliability is negated as shown in Fig. 64 (a), (b). The operation of the 
detecting regions 50, 51 is same as that of 48, 49, and the explanation is omitted. Next, as shown in Fig. 64 (b), when 
occupying the majority or whole of a certain detecting region 48, the motion vector 57 of the detecting region becomes 

20 a sum motion vector adding the motion of the moving object 52, and it is judged to be reliable from the state of the cor- 
relation value 55. Furthermore, at this time, the detecting region 49 is not reliable. Here, however, the detecting region 
49 is judged to be still in the motion vector by the still state judging means 43, and the motion vector 57 is ignored by 
the judgment result leading means 45, and the motion vectors of the detecting regions 49, 51 are used in the final deter- 
mination of the motion vector. As the moving object moves along with the passing of the time, the background and the 

25 moving object 52 are both present in the detecting region 48 again (this is (c) though not shown in Rg. 64), and the no- 
nliability state is established as shown in Fig. 64 (d), (e). In (c), (d), (e), however, since still vectors are present, the 
motion vectors detected at (a) to (e) in Fig. 65 are as shown by 59 in Fig. 65, so as not to be influenced by the motion 
of the object. 

Thus, according to the embodiment, if there is at least one motion vector judged to be still by the still state vector 
30 judging means, the still judgment result is given priority over the judgment result of the reliability judging means, and 
therefore if the reliability judgment means misjudges, it is finally possible to avoid output of wrong vector due to invasion' 
of the subject or the like. 

In the thirty-fifth embodiment, meanwhile, when there is a certain uniform brightness or color image such as the 
sky and the wall in a certain detecting region in the screen , if the screen swings due to camera shake or the like, the 

35 motion vector of that region obtained from the vector detecting circuit 42 of each region often delivers an uncertain 
value. At this time, if the still state judging circuit 43 judges that the uniform image is standing still, it is impossible to 
correct when the screen swings due to camera shake or the like. Accordingly, the still state judging means 43 may be 
further improved as follows. Rg. 66 shows its screen and the mean of the correlation value at that time. Numerals 86, 
87, 88, 89 are detecting regions. Rg. 66 (a) is a spontaneous scene, (c) is the mean at that time, (b) is the sky scene 

40 with uniform image detecting regions 87, 88, and (d) is the mean of the detecting regions 87, 88. The mean of the cor- 
relation values is obtained by obtaining the difference £|AL|(i,j) of the signaJ at the position deviated by a specified quan- 
tity (i.j) :imin < i <imax, jmin < j < jmax as the correlation value in each detecting region between two fields, and averaging 
the obtained values. Generally, when the screen is a spontaneous scene as in Fig. 66 (a), the mean of the detecting 
regions is a high value as in Fig. 66 (c). In the case of a uniform screen as in the detecting regions 87, 88 in Rg. 68 (b), 

45 on the other hand, the correlation values (the signal difference I|AL|(i,j)) become small in the entire region of imtn < i 
imax, jmin < j < jmax, showing smaller values than Fig. 66 (c), as indicated in Fig. 66 (d). When the mean is smaller 
than a specific value C\ it is enough when the detecting regions 87, 88 are judged to be not still. 

The still state judging means judges that the motion vector of the detecting region is still when the vector Vk and 
the mean satisfy the specified conditions consecutively 10 times or n times (n: a natural number) in the time series, for 

so example: 



Sk = (Vk(t) <C)n (Vk(M) < C} n{Vk(t-2) <C)n (Vk(t-3) < C) n 
{Vk(t-4) < C} n (Vk(t-5) < C} n{Vk(t-6) < C} n (Vk(t-7) < C} n 
(Vk(t-8) < C) n {Vk(t-9) < C) n (Avek(t) > C'J n {Avek(M) > C] n (Avek(t-2) > C'} n 
{Avek(t-3) > C'} n {Avek(t-4) > C'} n{Avek(t-5) > C} n {Avek(t-6) > C'J n 
{Avek(t-7) > C'} n {Avek(t-8) > C'} n (Avek(t-9) > C'} 
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where 



Sk: judgment result (1 if true, 0 if false) (k=1 ,2,3,4) 
t: time 

C, G': constants 

AND . . 0 " A x 

Vk- motion vector of each detectmg reg.on (k= ,2.3.4) 
Avek: mean of each detecting region (k=1 ,2,3,4) 



sixth embodiment ol theinvention. Numeral ^ ^'^^^JX judging means. 65 is judgment result leading 
of caC h reaion. 63 is minimum judging means, M«f»™"^» " 



of each region, ba is mimmu... ,~ a ■■ -- correcting means. 

detecting regions. H ^ judging mea ns 64 judges that the reliability is absent 

In the judgment result leading means 65 when ' ^ 9 & jg ^ and „ delivered t0 

at .east in one detecting region, the ^l^^ZZZ ^ 66. the minimum value among the motion 
the motion vector determining means 66. "n th« «ctt « , he re , ia bility judging means 

vectors of the detecting regions is taken th^^o^^nt resul^c^ the reliability judging means is employed, 
64 judges that al.detecting regions are re^^^ 
30 delivered to the motion vedordeterm,n^^ 

motion vectors of the detecting regions i« ith ^^"^^ep^o, step 111 corresponds to 62 in Fig. 67.^ 

shown in interval (a) in F,g. 64, when ^^ , ^^^™ on Jartateollhe correlation va.ue 53 as shorn 
region 48. that detecting region is |udged to be not °^£° rt|iabli M depending on the manner of repro- 
in Fig. 64 (a). It is desired that ^^^^J* - ^ JST «. Accordingly, the minimum judgment 
duction of the background, the operation of the detecting regions 50. 51 is same as 

40 result is employed by the judgment result taring JJJ^ Next , as shown in Fig. 64 (b). when occupying 
that of the detecting regions 48. 49 and ,s ha* .not ^^^^^^^ng region becomes a sum motion 
the majority or whole of the detecting re ^ n J%*^ 

vector adding the motion of the moving ob,ect52 andfcei rah*. « ^ ^ 49 » 

tion value 55. At this time, the detecting region 49 ' ■^JJ* 57 Ts ignored, and the motion vectors of the detecting 
« in the minimum value of the ^^^^ionT ^ Lion ^ecto Furthermore, when the moving object 52 moves 
regions 49. 51 are finally used in the determination of motor vector, w iQn ^ again {lnis 1S 



among ■ 

motion vector of the entire screen from n values. 
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Rg. 69 is a block diagram of swing correcting device of image including motion vector detecting device in a thirty- 
seventh embodiment of the invention. Numeral 68 is an image signal input terminal. Numeral 69 is a vector detecting 
circuit of each region, 70 is similarity judging means, 71 is vector region judging means, 72 is judgment result leading 
means, 73 is vector determining means, and 74 is swing correcting means. 
5 In thus composed thirty-seventh embodiment, the operation of the vector detecting circuit 69 of each region, relia- 

bility judging means 71 , vector determining means 73, and swing correcting means 74 are same as those of the vector 
detecting circuit 42 of each region, reliability judging means 44, vector determining means 46, and swing correcting 
means 47 in the thirty-fifth embodiment respectively. 

In the similarity judging means 70, the degree of similarity is calculated. To determine the similarity, the difference 
w portion of the motion vectors of the detecting regions is calculated. For example, it may be expressed as follows: 

Similarity = {|V1 1 - |V2|} + {|V1 1 - 1 V3|} + {|V1 1 - |V4|} + {|V2| - |V3|} + {|V2| - |V4|} + {|V3| - |V4|} 

where 

15 

Vi: (i = 1 , 2, 3, 4) motion vector of the detecting region, 
i is the number of detecting regions. 

The smaller the similarity, the more similar are the motion vectors of the detecting regions. In the similarity judging 
20 means 70 , when motion vectors of all detecting regions are similar, the result of judgment of all vectors is true (1 ). Oth- 
erwise, the result is false (0). The result of judgment, the degree of similarity, and motion vectors of detecting regions 
are delivered to the judgment result leading means 72. 

In the judgment result leading means 72, the judging means is selected on the basis of the similarity, 
if similarity ^ C, the similarity judgment result is used. 
25 if similarity > C, the reliability judgment result is used. 
C: constant 

This operation is shown in the flow chart in Fig. 70. Starting with step 1 19, step 120 corresponds to 69 in Fig. 69. 
step 121 to 71 in Fig. 69, step 122 to 70 in Fig. 69, steps 123, 124, 125 to 72 in Fig. 69, step 126 to 73 in Fig. 69, and 
a motion vector is delivered at step 127. 

30 A practical example is explained in Fig. 71, 72. Fig. 71 shows the screen before correction with camera shake, 
detecting regions 75, 77 and their correlation value 80, detecting regions 76, 78 and their correlation value 81. The 
operation of detecting regions 77, 78 is same as that of 75, 76, and the explanation is omitted. Numeral 79 is an obsta- 
cle. Fig. 72 shows the motion vector and similarity in Fig. 71. In Fig. 72, numeral 84 is a motion vector of camera, and 
it is swinging vertically about 0 vector. Numeral 83 is a motion vector detected from the detecting region 75, and the 

35 motion vector 84 due to camera shake or the motion of the moving object 79 or other wrong vector is detected. Numeral * 
82 is a motion vector detected from the detecting region 76, and the motion vector 84 due to camera shake is detected. 
Numeral 85 is a motion vector of the entire screen delivered from the vector determining means 73. Intervals (a) to (d) 
in Fig. 71 correspond to (a) to (d) in Rg. 72. 

First, in Fig. 71 (a) to (c), from Fig. 72, it is known that the state is 

40 Similarity ^ C 

and therefore the similarity judgment result is employed. By the vector determining means 73, the mean of all motion 
vectors in the detecting regions becomes the motion vector of the entire screen. 

In Fig. 71 (b), the reliability judgment is suddenly no reliability in all detecting regions due to the appearance(image 
signal ) of the background, and by this judgment alone, the swing is not corrected, but since the state is 
45 Similarity ^ C 

as known from Fig. 72, the similarity judgment result is employed. Next, in interval (d) in Fig. 71, the obstacle 79 has 
entered, and hence the state is 
Similarity > C 

and the reliability judgment result is employed. Hence, the motion vector of the detecting region 75 is employed. As a 
so result the motion vector of the entire screen is as shown in 85 in Rg. 72, and the vector correcting only the camera 
shake is obtained. 

Thus, according to the embodiment, when the motion vectors of the detecting regions are judged to be similar by 
the similarity judging means, the entire screen is swinging in a same direction, and therefore the screen swing must be 
corrected. Here, in the case on the basis of the similarity, motion vectors of all detecting regions are employed by the 
55 judgment result leading means 72, and are delivered to the vector determining means 73. To the contrary, if not similar, 
it is highly possible that an obstacle has entered the screen, and the detecting region containing the obstacle should 
not be used in correct on, and therefore the reliability judging means 71 is used. In such constitution, only the camera 
shake is corrected, and a stable and easy-to-see image is obtained. 

In the thirty-fifth, thirty-sixth and thirty-seventh embodiments, meanwhile, the number of detecting regions in the 
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screen is four but it is not limitative. Besides, the method of judging the reliability of the motion vectors of detecting 

Sv^ at that moment ^^^^ 
value or judgment result may be used together. 

Or, in the motion vector detecting device, all of the still state judging means, minimum judging means and s.mHarrty 

JKS^eE^ Numeral 131 is an image sfgna, input termina, Numer* 

circuit in each region. 133 is weighting processing means, and 134is swing correcting means, of which function issame 

secSfnSc^t leasmofieldsisfed into the input terminal 131. In the vector detecting circuit 132 of each region, 
assW.nF^ 

consecutive in time of two fields is entered. The difference from the signal at a position deviated by a specrfic quantity 
mv?rnT< i< imS in* < j < jmax between two fields wrth respect to representative points of each subd.vis.on ,s 
obla^ as meTo'rr'SonUe individually, and the sum of subdivisions is determined. The correlafcon value is 

obtained in the same formula as in the prior art. 

n 

Correlation value = £ | AL|(i,j) 
k=1 



25 where 



30 



S.|AL|(i,j) : absolute value of difference of signal between coordinates (i,j) and representative point 
n: number of subdivisions in one detecting region 

Detectina the deviation (i'j 1 ) for giving the minimum value of correlation values, it is taken as the motion vector Vk 
(k _ ? T 9 n n -^JJLA regions) of each detecting region. The motion vector of each detect.ng reg.on ,s 

coeflintme^ 

by a formula of weighting mean, for example, 

n n 

V- £(ak*Vk)/£ak 



so 



55 



where 



n: number of detecting regions 
ak: weight coeff icient 

Vk: motion vector of each detecting region 
V: motion vector of entire screen 

Two practical examples are shown in Fig. 74 (a), (b). Numerals 135 138 are detecting regions. 1 36 
vectors of detecting regions, 137. 140 are weight coefficients of detecting regions. In F,g. 74 (a), the screen .s drnded 
nto WetS regions, and in Fig. 74 (b), into 16 detecting regions. In both (a) and (b), U . **t coe«,c.en . 0 ,n 
the central part of the screen, and is larger at the end portions of the screen. Suppose, as shown ,n Fig 
IS. car) enters. Numerals 1 41 . 145 are detecting regions. 142. 146 are motion vector of detecting regions 
^TE£m**H** of detect regions, and 144. 148 are moving objecte (cars . * « ««JJjJj 
to F a 74 (a) and Fig 75 (b) to Fig. 74 (b). In Fig. 75 (a), the moving object is located nearly in the center S,nce the 
weSt co « ci^o?L cinira, detecting reg»n is 0. the motion vector of the moving object wi.l not «e motion 
vector of the entire screen. In Fig. 75 (b). the motion vector of the moving object is detected in reg.ons (V3 V6 V7, V10. 
STvS^S h?m. since the wei ht coefficient is 0 in V7, V7. V10, V1 1 . there will be no effect on the motion ve^ 
tor of tile entire screen. As for V3, V15, sines the value becomes relatively small due to we.ght.ng mean, and there ,s 
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no notable effect. 

Thus, according to the embodiment, the middle of the screen is the motion of the subject, and it is expected that 
the motion due to camera shake is present at the end portion of the screen, and when the weight coefficient. is set larger 
at the end portion of the screen. At this time, the central part of the screen may not be always 0. When the motion vector 

5 is weighted and averaged on the basis of the weight coefficient, a motion vector similar to the vector of the camera 
shake may be obtained by simple calculation. 

Fig. 76 is a block diagram of an image swing correcting device including a motion vector detecting device in a thirty- 
ninth embodiment of the invention. Numeral 149 is an image signal input terminal. Numeral 150 is a vector detecting 
circuit of each region, 1 51 is a reliability judging means 152 is weighting processing means, and 153 is swing correcting 

10 means having the same function as the swing correcting means 208 in Fig. 1 . 

In the image motion vector detecting device of thus composed thirty-ninth embodiment, first, an image signal con- 
secutive in time for at least two fields is entered into the input terminal 149. In the vector detecting circuit 150 of each 
region, same as in the thirty-ninth embodiment the motion vector is determined. As shown in Fig. 77 (a), plural vector 
detecting regions 155 are determined in the screen, and the detecting regions are further divided into multiple subdivi- 

15 sions having representative points explained in the prior art (Fig. 2). An image signal consecutive in time for two fields 
is entered. As for representative point of each subdivision, the difference from the signal at a position deviated by a 
specified quantity (i,j): imin < i < imax, jmin < j < jmax between two fields is determined as the correlation value, and the 
sum of the subdivisions is calculated. The correlation value is obtained in a same formula as in the prior art. 

20 n 

Correlation = £|Al_|(i,j) 



25 where 

lAL|(i,j): absolute value of difference of signal between coordinates (i,j) and representative point 
n: number of subdivisions in one detecting region 

30 Detecting the deviation (i'J) for giving the minimum of correlation values, it is determined as the motion vector of 

each detecting region (k = 1, 2 n; n: number of detecting regions). The vector detecting circuit 150 of each region 

determines the minimum, mean and maximum of the correlation values, and delivers them, together with the value of 
the motion vector of the vector reliability judging means 151. In the vector reliability judging means 151, from the mini- 
mum, mean and maximum of the correlation values of detecting regions, the reliability of the vector is judged. The reli- 

35 ability judgment result and the motion vector of the detecting region and delivered to the weighting processing means 
152. 

The weighting processing means 152 calculates on the basis of the weight coefficient and motion vector in the 
regions judged to be reliable by the reliability judging means 151. The weighting processing means 152 may be 
expressed, for example, in the following weighting mean expression. - 

40 

n n 

V= £(ak*Vk)/ £ak 

45 

where 

k: the number of the detecting region judged to be reliable 
(excluding the numbers not reliable) 
so n: number of detecting regions 
ak: weight coefficient 

Vk: motion vector of each detecting region 
V: motion vector of entire screen 

55 Fig. 77 (a), (b) show practical examples. Fig. 77 (a) is divided into 16 detecting regions, and the weight coefficient 
is given. Numeral 155 is a detecting region, 154 is a motion vector in each detecting region, 156 is a weight coefficient 
of each detecting region, and 1 57 is a moving object (a car). In (a), the motion vectors of the moving object (V3, V6, V7, 
V10, V1 1 V15) are detected. Of them, V6, V7, V10,' V11 are 0 in the weight coefficient and will not affect the motion 
vector of the entire screen. As for V3, V15, as shown in shaded area in (b), the reliability is negatively confirmed by the 
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15 



20 



«o« and thev will not be employed. Therefore, the weighting mean is calculated by the, t>oton 
reliability jodg.ng means 1 51 .and they w^ not b emp oy ^ ^ ^ entjre screen ^ detemned . 

vectors ot VI . V2. V4. V* V8, JW/W. V£V 6. an(j jt . s mat 

Thus, according to the embod.merrt, the rroddle of ^screen s ght coefficient is increased 

the motion by camera shake is preset* in the end f^^^S^ «* motion veCt0f ° f * § 
in the end portions of the screen ^^^J^^^^^ vector similar to the vector of the 

cir S h« 

tieth ertodiment of the invention. NumerallM-san^ 

. S *0«niMe5lnH,.79W,p!»S.^"« 

mined in the following equation, same as in the prior art. 



25 



Correlation value - £|AL|(i.j) 



where 

|AL|(i j) : absolute value of difference of signal between coordinates CM) and representative point 
n: number of subdivisions in one detecting region 



35 



40 



delivered to the classifying processing means 160. ri rteefina reaions There are many classifying tech- 

The classifying means 160 classes the motion ^^^"gj^ in this embodiment That is the 
niques. and any one may do, but an example of h.erarchical | 2SS^S^„7moto and magnitude of 
motion vectors are grouped secuentiaHy from 

: u r e rXo^ 

motion vectors between groups, for example, as shown below. 



Dmax = max | V i- V j I 

45 1 » J 



where 



50 



55 



Vi (i = 1 , 2, .... n): motion vectors in a certain group 
V] (j = i ' 2, m): motion vectors in a different group 
i, j: the numbers of detecting regions in a group 
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motion vectors in a group selected by the camera shake group judging means 1 61 . 

Fig. 79 (a), (b) are practical examples. Fig. 79 (a) divides the screen into 16 detecting regions. Numeral 164 is a 
motion vector of detecting region, and 1 65 is a detecting region. Suppose a moving object (a man) 1 66 enters as in (a). 
The man is walking while waving hand, from left to right of the screen. Therefore, the motion vector of the man is fluc- 
tuating As the motion vectors of the man. V5. V6. V7. V8, V9. V10, V1 1 , V12 are detected. On the other hand, as the 
camera shake motion vectors, VI. V2, V3. V4. V13. V14. V15, V16 are delected. Drawing these motion vectors by 
matching the start points of there , result in Fig. 79 (b). The motion vector is divided into two groups, as in (b). by using 
the classifying technique. Two groups are expressed in the detecting regions as shown in spotted area (group 1) and 
shaded area (group 2) in Fig. 80. Some of the detecting regions do not belong to any group. Which group should be 
used as the correct ion vector for camera shake may be judged by employing group 1 which is the motion vector before 
invasion of the man. 

Thus, according to the embodiment, by classifying (clustering) motion vectors in detecting regions into plural 
groups on the basis of the direction or magnitude or their differential value, and employing one of the groups as the 
motion vector of camera shake, a motion vector similar to the vector of camera shake may be obtained. 

Fig. 81 is a block diagram of an image swing correcting device including a motion vector detecting device in a forty- 
first embodiment of the invention, in which numeral 169 is an image signal input terminal. Numeral 170 is a vector 
detecting circuit of each region, 171 is reliability judging means, 172 is classifying mean, 173 is camera shake group 
judging means, 174 is motion vector determining means, and 174 is swing correcting means, which has the same func- 
tion as the swing correcting means 208 in Fig. 1 

In the image motion vector detecting device of thus composed forty-first embodiment, first, an image signal con- 
secutive in tine for at least two fields is entered into the input terminal 169. In the vector detecting circuit 170 of each 
region the motion vector is individually obtained. As shown in Fig. 82 (a), plural vector detecting regions 177 are pre- 
liminarily determined in the screen, and the detecting regions are further divided into multiple subdivisions having rep- 
resentative points explained in the prior art (Fig. 2). An image consecutive in time for two fields is entered. As for the 
representative point of each subdivision, the difference from the signal at a position deviated by a specified quantity (i.j): 
imin < i imax, jmin < j < jmax between two fields is determined as the correlation value, and the sum of the subdivisions 
is calculated'. The correlation value is determined, same as in the prior art, in the following equation. 

n 

Correlation = £|AL|(i,j) 



where 



|AL|(i,j) : absolute value of difference of signal between coordinates (ij) are representative point 
n: number of subdivisions is one detecting region 

Detecting the deviation (i'-jl for g ivin 9 tne minimum of correlation values, it is obtained as the motion vector of each 

detecting region Kv (k = 1. 2 n; n; number of detecting regions). The vector detecting circuit 170 of each region 

determines the minimum, mean and maximum of the correlation values, and delivers them, together with the value of 
the motion vector, to the vector reliability judging means 1 71 . In the vector reliability judging means 171, from the min- 
imum, mean and maximum of correlation values of each detecting region, the reliability of the vector is judged. The 
result'of reliability judgment and the motion vector of detecting region are delivered to the classifying means 172. 

The classifying means 172 clusters (classifies) the motion vectors of detecting regions judged to be reliable by the 
reliability judging means 1 71 . There are many classifying methods, and any one will do, and an example of hierarchical 
classifying is described in this embodiment. That is, those small in the distance between motion vectors (difference in 
direction and magnitude) are grouped together. Furthermore, those small in the distance between groups are merged 
to make up one group, and the number of groups is decreased. The distance between groups is expressed, for exam- 
ple, by the maximum distance among mutual motion vectors between groups as follows. 

Dmax = max | V i- V j | 
it J 



where 

Vi (i = 1. 2 n): motion vector in a certain group 
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Vj (j = 1 , 2 m): motion vector in other group 

i, j: the numbers of detecting regions in a group 



20 



25 



motion vectors within the group selected by the camera shake group judging means 173 1 

c- oo ,=\ im =h«« nrartical exarreles In Fig 82 fa), the screen is divided into 1 6 detecting regions, ana a moving 
Fig 82 (a) fl» "^Ste J 2 waving hand and enters the screen from left to right. As the melon 

object (a man) has invaded, man walte wh e waving nana ^ ^ Qf ^ shake , V1 , V2 , 

V9 VnO are lul * be J?i2SStha re.iabi.ity judging means 171 , classKying is effected by using the motion 
V9 V10 are judged tc i be : not re ao ^ classifying. The region of the camera shake is 

be selected. Bmhodim ertt bv classifying (clustering) only the motion vectors in the detecting regions^ 

^IT^^y^. wn E£EKl is automatical interrupted and detecting of wrong vector is pre- 

judging means, the performance is further enhanced. 



45 
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Claims 



An image motion vector detecting device comprising: 



32 



EP 0 789 487 A2 

means (3) for determining correlation values in specific deviation in each detecting region of disposed plural 
detecting regions in a screen, 

means (4) for determining a time variation value of the correlation values, 

means (2) for determining a motion vector in each detecting region from the correlation values, 

means (5) for judging a reliability of motion vector in each detecting region by using the correlation values and 
the time variation value, and 

means (6) for determining a motion vector of an entire screen by using the motion vector of each region on the 
basis of the judged reliability. 

I. An image motion detecting device comprising: 

means (2) for determining a motion vector in each detecting region of disposed plural detecting regions in a 
screen, 

judging means (8-10) for determining a judgment result having a meaning from true to false, from each one of. 
two or more pieces of reliability judgment information which is obtained by using correlation values in said each 
detecting region, 

judging value converting means (1 1 ) for converting the judgment result having a meaning from true to false into 
binary judgment value of true or false, and 

means (6) for determining a motion vector of an entire screen by using the motion vector of each region on the 
basis of the binary judgment value of each detecting region. 

3. An image motion vector detecting device comprising: 

means (2) for determining a motion vector in each detecting region of disposed plural detecting regions in a 
screen, 

judging means (5) for determining a judgment result having a meaning from true to false by a reliability judg- 
ment information which is obtained by using correlation values in said each detect.ng region, and 

means (6) for determining a motion vector of an entire screen by using the motion vector of each region on the 
basis of the judgment result having a meaning from true to false. 

4. An image motion vector detecting device comprising: 

means (2) for determining a motion vector in each detecting region of disposed plural detecting regions in a 
screen, 

means (3) for determining a correlation of motion vectors among the detecting regions. 

means (5) for judging a reliability of the motion vector in each detecting region by using the correlation values, 

means (20) for changing a judgment condition of the reliability judging means by a correlation value of motion 
vectors among detecting regions, and 

means (6) for determining a motion vector of an entire screen by using the motion vector of each region on the 
basis of the reliability judgment. 

5. An image vector detecting device in accordance with claim 4. wherein 

said means for changing a judgment condition changes the judgment condition made less severe when the 
correlation of the motion vectors among the detecting regions is high. 
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6. An image swing correcting device comprising: 

image signal memory means or CCD for at least one field, 
s an image motion vector detecting device of claim 1 , 2, 3 or 4, and 

reading position of the image signal memory means or CCD being controlled on the basis of a motion vector 
detected by the image motion vector detecting device. 

io 7. An image motion vector detecting device comprising: 

means (23a-d) for determining a correlation value in a specific deviation in each detecting region of disposed 
plural detecting regions in a screen, 

is means (25, 26) for determining a difference of correlation values between right and left, or upper and lower 

detecting regions in a screen, 

means (22a-d) determining a motion vector in each detecting region on the basis of the correlation values, 

20 means (24a<J) judging a reliability of the motion vector in each detecting region on the basis of the correlation 

value in each detecting region, and 

means (27) determining a motion vector of an entire screen by using the motion vector of each detecting region 
on the basis of the difference of the correlation values between regions and the judgment of reliability. 

25 

8. An image motion vector detecting device comprising: 

means for determining a correlation value in a specific deviation of each detecting region of disposed plural 
detecting regions in a screen, 

means for determining a difference of correlation value between right and left, or upper and lower detecting 
regions in a screen, 

means for determining a motion vector in each detecting region on the basis of the correlation values, 

means for judging a reliability of a motion vector in each detecting region on the basis of the correlation value 
in each detecting region and the difference of correlation values between regions, and 

means for determining a motion vector of an entire screen by using the motion vector of each detecting region 
40 on the basis of the judgment of the reliability. 

9. An image motion vector detecting device comprising: 

means (23a-d) for determining a correlation value in a specific deviation in each detecting region of disposed 
45 plural detecting regions in a screen, 

means (30a<f) for determining a time variation value of correlation values, 

means (32, 33) for determining a difference of the time variation values of the correlation values between right 
so and left, or upper and lower detecting regions in a screen, 

means (22a<f) for determining a motion vector in each detecting region on the basis of the correlation values, 

means (31a-d) for judging a reliability of motion vector in each detecting region on the basis of the correlation 
ss value in each detecting region and the time variation value of the correlation values, and 

means (34) for determining a motion vector of an entire screen by using the motion vector of each detecting 
region on the basis of the judgment of reliability and the difference in time variation value of correlation values 
between regions. 
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